A portion of the disclosure of this patent document contains material to which a claim of copyright protection is made. The copyright owner has no objection to the facsimile reproduction of any one of the patent documents or the patent disclosure, as it appears in the U.S. Patent and Trademark Office patent file or records, but reverses all other rights whatsoever.
A time-varying image sequence is a sequence of images of a given scene with each successive image taken some time interval apart from the one preceding it. If the scene being imaged changes gradually with time and, if the changes are mostly due to the relative movement of the physical objects in space, then the corresponding changes in the successive images of the sequence can be characterized by velocity vector fields.
The reliable estimation of the velocity vector fields is very important for the analysis of time-varying image sequences. There are two principal approaches to the problem of estimation of the velocity vector field: the feature-based matching approach and the spatio-temporal gradient approach. In the feature-based matching approach, the image points with significant variations in the values of the time-varying image functions, called feature points, are identified in both images. The estimation of the velocity vector field is accomplished by matching the feature points of one image to the feature points of the other image. The spatio-temporal gradient approach is based on the constraint imposed on each velocity vector relating the spatial gradient of the time-varying image functions to the temporal derivative of the time-varying image function.
The spatial variations in the time-varying image function, utilized in the above approaches, do not provide sufficient information to determine the estimate of the velocity vector field. If the first approach, the velocity vectors can only be estimated on a sparse set of image points, while in the second approach, at most one constraint is imposed on two components of each velocity vector. To overcome these difficulties, it has been proposed that velocity fields should vary smoothly from point-to-point on the image plane. This requirement enables the estimation of both components of the velocity vector at each image point; however, it forces the estimate of the velocity vector field to vary smoothly across the occlusion boundaries. Several approaches have been proposed to overcome this difficulty, which are based on the selective application of a smoothness requirement.
In the present method, the estimate of the velocity vector field is determined as a compromise in the attempt to satisfy the following two sets of constraints in addition to the regularization constraints; the optical flow constraints, which relate the values of the time-varying image function at corresponding points of the successive image of the sequence, and the directional smoothness constraints, which relate the values of the neighboring velocity vectors.
These constraints are selectively suppressed in the neighborhoods of the occlusion boundaries. The last in accomplished by attaching a weight to each constraint. The spatial variations in the values of the time-varying image function near corresponding points of the successive images of the sequence, with the correspondence specified by a current estimate of the velocity vector field, and variations in the current estimate of the velocity vectors themselves are implicitly used to adjust the weight functions.